Complex systems are all around us. From weather to ecosystems to biological systems to information technology systems, various tools have been developed in attempt to study and understand these systems in detail, mainly through the use of computer-analyzed data. Using computers, information may be analyzed and presented for human consumption and understanding, even when dealing with complex systems. These computerized tools allow variables in complex systems to be analyzed ranging from biological systems, traffic congestion problems, weather problems, information technology systems and problems, and complex systems that would otherwise be too information rich for human understanding.
As the volumes of information regarding complex systems have increased, the tools for efficiently storing, handling, evaluating, and analyzing the data have evolved. For all of their power in accomplishing these tasks, however, existing tools to date suffer from an inherent inability to predict future events with accuracy in these complex systems. The problem with prior approaches is that computers are only capable of performing the tasks they are programmed to analyze. Consequently, in order to have a computer evaluate and predict outcomes in complex systems, computers must be programmed and instructed how to make the predictions. However, humans must first have a core understanding of the variables at play in a complex system in order to tell the computer how to predict outcomes. In most cases, the human operators are not able to instruct the computer how to model each variable in a complex system with enough precision to tell the computer how to make the predictions. Moreover, many variables in complex systems exhibit behavioral changes depending on the behavior of other variables. Thus, what is needed is a tool that allows humans to evaluate the variables without a complete understanding of every variable at play. In other words, there is a need to establish the ability to generate sui generis truth rules about predicates outside of the limitations of human consciousness and awareness.
Of particular importance are variations that occur within complex systems, such as abnormalities or problems. For instance, in the case of an information technology (IT) infrastructure, these variations from normal or expected operation could lead to failures, slowdown, threshold violations, and other problems. Often, these types of problems are triggered by unobserved variations or abnormalities in one or more nodes that cascade into larger problems until they are finally observable. Prediction of these of variations from the expected can require an understanding of the underlying causes, which may only be observable by accounting for the behaviors of the variables in substantially real-time. Moreover, the more remote in time from the event leading to the variation, the more sensitive the relevant analysis must be to detect the underlying causes. As a result, many tools currently used to address abnormalities in complex systems work close to the time the problem is actually observed to humans. Other tools are based on inefficient thresholding systems that address the potential abnormality when the probability that the abnormality will lead to an actual problem remains small, which triggers responses in many cases where a response is not actually merited.
Existing thresholding systems provide an inefficient and ineffective tool for predicting problems before they actually occur within enough time to address the underlying abnormality so as to be able to prevent the problems, because these systems are generally incapable of differentiating between a minute deviation from normal operational behavior (e.g., a “spike”) and deviating trends, which are often indicative of an abnormality. However, because these systems do not address the causes and are activated in close time proximity to actual abnormalities, implementers often set the sensitivity of thresholding systems to be very sensitive, which often produces false-positive abnormality alerts and consequently creates inefficiencies in troubleshooting and addressing the abnormalities.
For abnormalities of nodes in complex systems, however, the sheer number of variables makes prediction of the abnormalities difficult. There is a need for a heuristic that utilizes thresholding functionality to provide a more efficient system for prediction and alleviation of abnormalities before they lead to problem events, without producing high rates of false positive abnormality alerts.